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ABSTRACT 


A  simple  time  series  model  foi 
having  first-order  autoregressive  st 
random  coefficient  difference  equatioi 
Exponential  Autoregressive  model  (NE/ 
in  the  bivariate  sense  and  has  cor  re 
of  the  Gaussian  AR(1)  bivariate  time  : 
full  range  of  positive  correlations  j 
modification  in  either  the  innovatio 
model  admits  some  negative  values 
marginal  processes  are  shown  to  have 
mode 1 s . 


bivariate  exponential  variables 
ructure  is  presented.  The  linear 
i  model  is  an  adaptation  of  the  New 
R  (2)).  The  process  is  Markovian 
lation  structure  analogous  to  that 
series  model.  The  model  exhibits  a 
and  cross-correlations.  With  some 
n  or  the  random  coefficients,  the 
for  the  cross-correlations.  The 
correlation  structure  of  ARMA(2.1) 


Key  Words: 


Time    Series; 
Autoregressive  Mc 
AR(1)  Bivariate  1 


Distribution; 
>dels;  Gaussian 


1 .   INTRODUCTION 

The  homogeneous  Poisson  process 
(series  of  events)  and  can  be  char 
intervals  between  events  are  indepi 
distributed  random  variables. 
Poisson  process  has  been  to  rela: 
intervals.  The  earliest  attempts  t 
Cox  (1955).   See  Cox  and  Lewis  (19* 


ic  model  for  point  processes 
1  as  a  process  in  which  the 
id  identically  exponentially 
lization  of  the  homogeneous 
ndition  on  independence  of 
s  go  back  to  Wold  (1948)  and 
)  for  details.   Later,  this 
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need  for  point  processes  with  correlated  but  marginally  exponential 
distributions  inspired  the  creation  of  a  class  of  ARMA  time  series 
models  for  the  univariate  exponential  processes  analogous  to  that  for 
the  Gaussian  processes  (Gaver  and  Lewis,  1980;  Lawrance  and  Lewis,  1981, 
1985).  Beyond  their  utility  in  point  processes,  these  models  are 
utilized  in  studying  queues  (Jacobs,  1978;  1980),  inventory  and  water 
resources  problems,  and  other  situations  where  non-negative  random 
variables  are  appropriate  for  inputs.  Using  antithetic  variables  as 
innovation  or  error  structure,  these  models  also  allow  for  negative 
dependence  (see  Lewis,  1986,  for  a  survey). 

We  demonstrate  that  one  of  the  more  recent  exponential  models  with 
ARMA  correlation  strucutre,  N£AR(2),  by  Lawrance  and  Lewis  (1985),  lends 
itself  naturally  to  a  first-order  bivariate  autoregressive  process  which 
is  Narkovian  in  the  bivariate  sense,  and  has  a  structure  analogous  to 
that  of  the  Gaussian  AR(1)  bivariate  time  series  model  as  given  in  Tiao 
and  Box  (1981),  and  elsewhere.  Early  attempts  to  construct  bivariate 
exponential  time  series  and  bivariate  Poisson  processes  are  discussed  in 
Cox  and  Lewis  (1971).  The  present  model  is  broader  and  simpler  than  any 
previously  obtained,  including  the  MEAR(l)  model  by  Raftery  (1982). 
which  turns  out  to  be  a  special  case  of  the  present  model  for  the 
bivariate  case,  and  the  models  of  Jacobs  (1978,  1980),  Lewis  and  Shedler 
(1977),  and  Gaver  and  Lewis  (1980).  All  of  these  models  were  defined 
using  the  NEAR(l)  structure,  and  this  does  not  allow  for  the  breadth 
obtained  by  using  the  NEAR(2)  structure. 

With  the  general  model  it  is,  for  example,  possible  to  construct 
simple  bivariate,  serially  correlated  models  for  the  successive  up  and 
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down  times  which  occur  in  reliability  studies,  or  for  correlated  and 
cross-correlated  service  and  interarrival  sequences  in  queueing  systems. 

2.  THE  UNIVARIATE  NEAR(2)  MODEL 

By  the  NEAR(2)  theorem  (Lawrance  and  Lewis,  1985).  we  know  that  two 
(possibly  dependent)  random  variables  with  marginally  Exponential (X) 
distributions  can  be  combined  with  three  scaled  (possibly  dependent) 
exponentials  which  are  independent  of  the  first  pair,  to  give  another 
random  variable  that  is  marginally  exponentially  distributed  with 
parameter  X.  We  assume  in  the  remainder  of  this  presentation  that 
X  =  1.  The  essence  of  the  NEAR(2)  theorem  is  repeated  without  proof. 

THEOREM  (Lawrance  and  Lewis) 
If   {E  }   is  an  i.i.d.  sequence  of  exponential  random  variables  and  the 
parameters   a~ ,  /3. ,  a^,  P«   are  such  that   0  <  a.  <1   for  i=l,2; 
0  <  a.   +   a2  <  1  and  0  <  p..   p~  <  1     and  if  XQ  =  EQ,   then 


X  = 

n 


PlXn-l   Wp-  al 


P2Xn-2   Wp-  a2 


w.p.   1-a, ~an 
r  12 


n 


w.p.   l-p0-p 


2  **3 


b2En    w.p.  p2 


b3En    wp'  P3. 


(2.1) 


defines  a  stationary  sequence  {X  }  which  is  marginally  exponentially 
distributed  with  X=l .  The  quantities  b^,  b3,  p2>  p^  have  values  in 
the  interval  (0,1)  and  are  defined  in  Lawrance  and  Lewis  (1985);  the 
mixture  of  the  exponential  random  variables  on  the  right  in  (2.1)  is 
called  the  innovation  random  variable. 
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A  special  case  of  the  N£AR(2)  model  which  is  not  covered  by  the 
theorem,  but  obtained  directly  from  the  usual  analysis  with  transforms, 
is   the  case  when    p\  =  0rt  =  1.    In  this  case,   still  assuming 


"l  =  "2  =  »• 


al  +a2.   *  lf  We  nave 


X  = 

n 


n-1 


n-2 


w.p.  a, 


w.p.  ar 


p.   l-a!-«2 


+  H-a.-a^E^ 


(2.2) 


This  is  the  TEAR(2)  model.  It  has  two  advantages  for  our  purposes;  one 
that  it  only  uses  two  parameters  Instead  of  four  in  the  complete  NEAR(2) 
model;  and  the  other  that  it  has  a  simpler  innovation  structure.  The 
model  exhibits,  however,  a  "runs-up"  behavior  in  the  sample  paths. 
Nevertheless,  we  will  use  the  TEAR(2)  model  as  the  base  for  construction 
of  the  bivariate  exponential  model,  even  though  other  three-  and 
two-parameter  subclasses  of  the  NEAR(2)  process  may  be 
phenomeno logically  more  suitable.  For  example,  in  hydrology  the  time 
series  which  occur  generally  exhibit  a  "runs-down"  behavior.  However, 
the  ideas  in  developing  the  bivariate  process  are  the  same. 

3.  BIVARIATE.  HARROVIAN  TIME  SERIES  FOR  EXPONENTIAL  RANDOM  VARIABLES 
Let  {X  ,Y  }  be  a  bivariate  sequence  of  random  variables,  such  that 


{X  } 
1  n' 


and 


{Y  }    are  each  stationary  sequences  of  marginally 


exponentially  distributed  random  variables  with   X=l.   Then  we  define 
our  first  bivariate  exponential  model,  using  (2.2).  as 


X  = 

n 


KU)Xn-l  +  K12)Yn-l  +  (^ll-l^n 


(3.1) 
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and 


Yn  =  «*     X„-l  +  4S)Y„-1  +  ^""a^l'V 
in  this  equation 


(3.2) 


{Kn  .  K12  )  - 


(0.1) 

w.p. 

a12 

(i.o) 

w.p. 

all 

(0.0) 

w.p. 

1-all~a12 

and  are  serially  independent; 


(0.1) 

w.p. 

a22 

(K21    ' 

k£>>  = 

(1.0) 

w.p. 

a21 

(0.0) 

w.p. 

1  "~tt_  .. —ct,-..-. 
21     22 

all  +  a12  <  * 


are  also  serially  independent  and  independent  of  {KL  .  KJ }  for  all 
n.  Likewise  {E  }  and  {E*}  are  independent  innovation  sequences  of 
i.i.d.  Exponential (X=l)  random  variables.  Finally,  we  insist  that 
and  a99,  +  a^*  <  1.  If  an  +  a\o  ~  *  an<^ 
a22  +  a21  =  *'  tnen  tne  process  is  not  ergodic.  In  this  case,  {X  ,Y  } 
is  always  one  of  the  pairs   (E-.EJ),  (E..E. ),  (E'E')  (E'E-)   for  all 

n.   Note  that   X  ,   and   Y  ,   in,  e.g..  Equation  (3.1)  are  not 

n-1         n-1        &  v   ' 

independent;  this  is  the  reason  for  the  use  of  the  NEAR(2)  construction. 
We  observe  that  {X  },  {Y  }  are  each  TEAR(2)  constructions  in  that 
Y  j  in  (3.1)  replaced  X  j  in  (2.1).  and  similarly  Xnl  in  (3.2) 
replaced  Y  _t>.  Also,  (X  ,Y  )  is.  given  the  value  of  (xn_i  •*  i ). 
completely  independent  of  previous  values   (^1,'^u)   *°r   ^  ^  n~^   f°r 
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all  n.   Again  £(X  |X ,  =  x,  Y ,  =  y)   is  a  linear  function  of   (x.y) 

and  likewise  for   Y  .   Thus,  the  model  is  Markovian  in  the  bivariate 

n 

sense,  both  structurally  and  in  the  sense  of  expectations. 

Note,  however,  that  {X  }  and  {Y  }  are  not  marginally  TEAR(l) 
processes,  and.  in  fact,  are  not  marginally  Markovian.  In  particular, 
their  correlations  are  not  necessarily  geometrically  decaying,  as  will 
be  demonstrated  later.  If  a..  =  a~n  =  0,  we  obtain  the  bivariate 
exponential  process  described  by  Gaver  and  Lewis  (1980).  The  process 
{Y  }   is  then  called  the  dual  of  the  process  {X  }. 

Clearly,  other  special  combinations  of  the  a  give  other  models 

with  1,  2,  or  3  parameters.  Of  course,  there  could  be  from  5  to  8  total 
parameters  by  using  the  more  general  NEAR(2)  construction.  Thus,  from 
(2.1).  (3.1).  and  (3.2).  replace  K^  by  0  K^ .  0  <  /3  <  1.  One 
requires  then,  however,  the  more  complicated  innovation  structure  given 
in  (2.1)  for  the  innovation  in  (3.1)  and  (3.2). 


4.  CORRELATION  STRUCTURE  OF  THE  MODEL 

The  marginal  processes  defined  above  are  exponentially  distributed 

2 

with  mean  1/A  and  variance   1/X  .   Thus,  if  we  set  the  scale  parameter 

X   equal  to  1,  correlations  and  covariance  are  equal.   Since  we  assume 
(Y  }.  {X )  are  jointly  stationary,  we  have 


t^(e)  =  i(xnxn_e)  -  i.   e  =  o.  1.  2.  •••.     (4.1) 

nr^O  =  *(YnYn_p  -  1.    £  .  0.  1.  2.  —.      (4.2) 

for  the  autocovariance  functions.  Similarly, 
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f^i*)   =  «(XnYn_p  -  1    €   =  0.  ±   1.  ±  2.  •••.    (4.3) 

is  the  cross-covariance  function. 

The  following  results  are  immediately  obtained  using  (3.1),  (3,2), 
(4.1)  -  (4.3)  and  the  stationary  assumption: 

-rxx(O)  =  ^(0)  =  1;  (4.4) 

nrXY(0)  =  nrYX(0)  =  (a22a12  +  "ll0^7*1  "  (a22all  +  a21a12)};  (45) 
^(5)  =  ^xx(-^)  for  all  «:  (4.6) 

i^ii)   =  -Ty^-*)  for  all  e-.  (4.7) 

ty^ii)   =  -*yx(~*)   for  a11  €l  (4-8a) 

^YX(«)  =  ^XY^)  for  a11  '*  (4.8b) 

Autocovariance  functons  are  even  functions  of  the   lag  £,       but  the 
cross-covariance  is  not. 

The  range  of  ^wC^)  =  ^yy^  as  8iven  in  (4-5)  is  the  full  range 
of  non-negative  values  up  to  one.  Using  the  Generalized  Reduced 
Gradient  (GRG)  algorithm,  we  find  that  the  maximum  takes  place  along  the 
boundary  a..  +  a.~  +  fe  =  1  and  a^  +  a^.  +  e  =  1  for  arbitrary  e  >  0. 
in  which  case  we  obtained  covariances  as  close  to  one  as  desired.  Of 
course,   for  strictly  positive    a.  that  satisfy  the  original 
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constraints.   1^(0)   cannot  be  negative.   We  show  in  Section  5  how  to 
incorporate  negative  correlations  into  the  model. 

Using  Equations  (3.1),  (3.2).  (4.1)-(4.8)  and  the  stationary  assumption, 
we  obtain  the  following  recursion  equations  for    ^yy^)'    "Vy^)* 


•W**  =  "lrxx^"1) +  ^VX*'"1* 


i   =  1.2. 


(4.9) 


-r^i)   =  o^twit-l)   +  a-.-rvvil-l) .  i   =  1.2.  —  .      (4.10) 


'YYX(')  =  a2rXX(*_1)  +  ^YX^-1) 


e  =  i,2.---. 


(4.11) 


'VyO  =  aio^wC^-1)  +  ttn^YvC*-1)'     e   =  1>2'" 


XY' 


12  YY 


11  XY' 


(4.12) 


It  remains  to  show  that  these  equations  correspond  to  those  given 
by,  for  example.  Tiao  and  Box  (1981),  for  the  AR(1)  Gaussian  bivariate 
time  series.  We  introduce  the  following  matrix  notation: 


ne)  = 


^xxO    -*nW 


'W*     'YYY(5) 


e   =  0.1.2,"-.    (4.13) 


is  the  covariance  matrix  for  lag  £.     Equations  (3.1)  and  (3.2)  become 


X  =  K  X  ,  +  AE  . 

n    n  n-1     n 


(4.14) 
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where 


A  = 


1-an-ai2 

0 

0 

1"a22"0C21   . 

t 

X     = 
n 

n 


n 


and 


K  = 
n 


E  = 

n 


n 


E' 
n 


K 


(n)     v(n) 


11 


K 


12 


^1 


K(n) 
*22 


The  matrix-valued  random  variable    K     assumes  9  different 

n 

matrix-values  independent  of  X  _1  and  E  and  retains  the  property  that 
«(K?nb  =  ai       for  all  i.  j.  n.  Then 


<T(Xn)  =  «(Kn)«(Vl>  +  A  «(En) 


(4.15) 


all 

a12 

*  1 

+ 

'  l-an-a12             0 

'  1 

"  1 

a21 

a22  . 

1 

0              1-ot22"a21 

1 

1 

Also, 


r(0)  =  «(x  xT)  -  i(x  )*(xT)  =  £(%  xT)  - 

v  '  x   n  n'    v  n'  v  n'    v  n  n' 


1  1 
1  1 


=  5{X  (K  X  ,  +  AE  )  }  - 

v  nv  n  n-1     n'  ' 


1   1 
1   1 


5{X  (K  X  JT)  +  «{X  (AE  }T}  - 
x  nv  n  n-1'  '    v  nv  n'  ' 


1  l] 
1  Ij 
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■  <«KnXn-l  +  'Vltt-lrt  +  «<KnXn-l  +  *V<*VT>  "  [}  }] 


Thus 


1{K  X  ,(KX  J7}  +  «{AE  (K  X  J7}  +  «{K  X  ,  (AE  )T> 
1  n  n-1  v  n  n-1'  '  l  nv  n  n-1'  '    l  n  n-1  v  n'  ' 


i(*^>T>  -  [I  }]. 


T(0)  =  i(K  X  ,X7  ,  K7)  +  «(AE  X  ,K7)  +  *(K  X  ,E7A7) 
v  '    v  n  n-1  n-1  n'    v  n  n-1  n'    v  n  n-1  n  ' 


jeJa7)  -[j  }] 


n  n 


(4.16) 


Solving  (4.16)  yields  the  results  given  by  Equations  (4.4)  and  (4.5) 

Furthermore 


ni) .  xvO  -  «V»CI>  ■  <«X-i> 


-II :] 


=  i{(K  X     ,   +  AE  )X7  ,}  -    f}     }] 
vv  n  n-1  n'  n-1'        [1     1J 


«(Kn)r(0)  +  «(Kn) 


1  1]  ♦  -cvti>  -  [1  1] 


Thus 


r(i)  =  «(Kn)r(o)  = 


all       a12 


a21       a22 


r(0). 


(4.17) 
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Again 


r(-i)  =  «(x  x'    )  - 

v      '  v  n  n+1' 


1  l\ 


=  «VKn+lX»  +  "W^  -    [1     !] 
=  «XnXnKn*l>  +  '(XnELl)AT  "    [1     1] 


Thus 


r(-i)  =  r(o)«(K^+1)  =  r(i)T. 


(4.18) 


Clearly  T(-l)  =  T(l).   Because  of  (4.8a.b)  and  (4.13).  we  have 


H-e)  *  ney 


(4.19) 


and 


He)  = 


all   a12 


a21   a22 


T(0)  for  all  €   =  1.2, 


(4  20) 


Now  Equation  (4.20)  is  the  result  given  by  Tiao  and  Box.  (1981. 
p.  804).  Therefore,  the  auto-  and  cross-covariances  in  this  bivariate 
exponential  process  in  general  decay  gradually  to  zero  as  |^| 
increases,  for  the  values  of  a  that  satisfy  the  conditions  in 

Section  3.   Numerical  examples  are  given  in  Section  6,  and  further 
discussion  of  the  auto-correlations  is  presented  in  Section  7. 
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In  principle,  one  can  use  standard  estimates  of  auto-  and 
cross-correlations  (Kendall  and  Stuart.  1983.  pp.  661-662)  to  obtain 
estimates  of  T(0)  and  T(l)  as 


C(0)  = 


i    Sw«>) 

0^(0)     1 


(4.21) 


and 


C(l)  = 


See*1*  <XiM 

0^(1)     Cyy 


(4.22) 


Then  (4.20)  yields  estimates  of     a..,   a^. ,   0N«.   a-<y>  as 


Lll 


l21 


12 


'22  J 


=  <C(1)}{C(0)} 


-1 


(4.23) 


Like  most  moment  estimates,  these  are  likely  to  be  fairly  poor  for  small 
sample  size. 

An  example  is  given  in  Table  4.1  where  we  have  performed  a 
simulation  of  the  estimation  of  a. . .  a19,  a~.  and  a„„  based  on  (4.23). 
Here  10  replications  were  used  and  the  true  values  were  a.  -  =  0.70; 
a-2  =  0.20;  a~.  =  0.30;  a**  =  0.50.  The  10  series  were  run  out  to 
n  =  600  and  the  first  100  values  were  discarded  as  being  nonstatlonary. 
The  boxplots  in  the  tables  show  the  distribution  of  the  tabulated  values 
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in  the  columns.    The  estmates  show  no  evidence  of  departure  from 
normality,  although  the  number  of  replications  is  small. 

TABLE  4.1 

Replication  an   =  0.70    a12  =  0.20    a21  =  0.30    a^  =  0.50 


.•3 


.31 


SO 


.7; 


.71 


.72 


it 


.74. 


,es 


.  «e 


.  70 


0 


.  17 


li 


o» 


23 


.22 


.0* 


23 


.2* 


.23 


.38 


.2* 


.23 


HB 


.ife 


.S3 


T 


.33 


.33 


.32 


38 


90 


.18 


.30 


5.  NEGATIVE  DEPENEENCE 

As  pointed  out  in  the  last  section,  the  first  bivariate  exponential 

model  discussed  above  can  produce  only  positive  correlations.   Negative 

correlation  is  obtainable  in  one  of  two  ways  -  through  correlation  in 

the  bivariate  innovation  {E  ,E'}  and/or  through  the  correlation  in  the 

attenuation  controlled  by  the  K^ .  .  These  ideas  are  relaxations  of  the 

conditions  established  in  Section  3  where   {E  }  and   {E'}  are  assumed 

1  n1       l  n' 
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to  be  i.i.d.  exponential  sequences  and  (Ki?  •  K:^  }  is  an  independent 
identically  distributed  bivariate  sequence  independent  of  {KiA  KJg  / 
for  all  n. 

It  is  easy  to  see  that  these  schemes  affect  directly  only  the  zero 
lag  covariances,  T(0) .  and  that  (4.20)  still  holds.  Thus,  it  is  only 
necessary  here  to  derive  the  effect  on  T(0)  of  correlation  in  the 
innovation  or  attenuation  sequences. 

5.i.  Correlated  Innovations. 

As  a  first  example,  consider  the  model  of  Section  3  and  let 


°»<*n-KnJ   = 


a       k=0 

(5.1) 
0   otherwise. 


Now  Equation  (4.5)  becomes 


fm       (allg21+g12g22>  *   <1-gll-g12»1-g21-a22>g   ,.  „, 

VW 1  -  («lrtB«liAi> '    (52) 


Moran  (1967)  showed  that  if   {E  }   and   (E'}   are  pairs  of  exponential 

random  variables  with  X  =  1.   then  -.6449  <  a  i   1.   If  E'  =  a(E  ).   the 

n    v  n' 

antithetic  transform  of  E  (see  for  example  Gaver  and  Lewis.  1980), 
then  a  takes  on  its  maximum  negative  value.  It  is  clear  that  if 
a  <  0,  then  (5.2)  could  be  negative  for  some  choices  of  the  a.  's. 
The  process  cannot  be  uncoupled  as  described  previously. 

Estimation  of  a  proceeds  as  before.  The  parameters  a..,  a.„, 
a21*  a22  are  estimated  by  (4.23)  and  then  (5.2)  can  be  easily  solved 
for  an  estimate  of  a. 
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5.ii.  Correlation  in  the  Random  Coefficients. 

Consider  now  the  model  of  Section  3  in  the  form  of  (4.14).   K 


n 


>(n) 


defines  a  quadruple  where  each  K)  .'   has  a  marginal  distribution  and  each 

ij 

row  has  an  independent  bivariate  distribution  for  each  n.  If  K  is 
now  defined  so  that  the  rows  are  not  stochastically  independent  of  each 
other,  then  (4.5)  becomes 


,M       {(allq214ql^a22)  +  ag11.21*ag12.22i,*<y11.22W12.21>   ,.  ,, 

^yy(°)  = 1  _  /„  ~  ^  „ — rr-^ — •  (5-3) 


XY' 


1  "  ^a\la22^a\2a2\)  ' 


11.22 


-  a 


12.21 


where 


ij.An 


Cov{K£>.Kirk)> 


Bra 


k  =  0 


otherwise. 


(5.4) 


This  correlation  must  be  introduced  in  such  a  way  that  the  marginal 
distributions  are  not  altered.  This  may  not  be  easy. 

The  following  example  using  KJn'  =  Ki^'   for  all   n  demonstrates 


the  above  ideas.   Define  K   as  follows: 

n 


„(n)  „(n) 
Kll  K12 

Un)  K(n) 
*21  Kll 


=  K  = 

n 


1 

0 

0 

1 

w.p. 

alV 

0 

,1 

a 

w.p. 

a21a12* 

0 
0 

S] 

w.p. 

a12(l-a21).                          (5.5) 

[? 

a 

w.p. 

a21(l-a12). 

o 
.0 

o1 

0. 

w.p. 

1  "  ("ll+^^l-0^!)' 
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where  a^.  al2,  a21  as  before,  and  also  0  i  an+ai2+a2i~ai2<121  *  *' 
It  is  easily  verified  that  the  marginal  distributions  of  the  quadruple 
are  retained  and,  specifically,  that 


«oy  - 


all   a12 


a21   all 


(5.6) 


For  K   defined  by  (5.5)  we  have,  from  (5.3) 


fM       gll<1~^all4a124g21» 


(5.7) 


This  follows  because     on21  =  -an^;   a12  ^  =  -fl^jj!   c^  ^ 


aii(1_an)   and 


=  0.   with   0  £  ot1+a10  £  1;  0  <[  an+a01  £  1; 


n\-  -n/        ~12.21  "  "•   w  ■»  ~11'~12  **•-■»  -ii  "21 

0  £  an+aio+a21~a12a21  ^  *"  Using  again  GRG,  we  find  ^yvC^)  attains 
the  full  range  of  positive  correlations  as  before.  It  has  a  max 
negative  correlation  of  -.125  at  o..  =  .25,  a-„  =  a?1  =  .5.  This 
corresponds  to  one  of  the  cases  where  K  /   0  for  any  n. 

Considering  the  bivariate  model  with  K  given  by  (5.5)  and  with 
a..  =  0,  we  see  that  *r.-.(2k)  =  0  =  -rYX(2k).  k  =  0,1.2,*»».  Resolving 
(4.9)  -  (4.12)  yields 


T(k)  = 


12 


*21   ° 


k  =  1.2. 


and 


r(0)  = 


1   o 
o   1 
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This  Is  the  correlation  structure  of  the  dual  process  of  Gaver  and  Lewis 
(1980).   No  negative  correlation  Is  possible  In  this  configuration. 

Lewis  and  Lawrance  (1981)  considered  the  model  with  correlation  in 
the  random  coefficients  and  made  k\o  aii<^  ^91  have  the  maximum 
possible  negative  correlation,  obtaining 


V(0)  =  T 7-TZ T  <  0, 


where 


a  =  Cov(k£>.  k£>)   =  a12    <  0. 


Considering  the  bivarlate  model  with   K    given  by  (5.5)  with 

°19  =  a9l  =  ®*      tnen  'VvCO)  ~  an  an<^  tne  reso^ut^on  °f  (4.9)  -  (4.12) 
with  (5.7)  yields  the  following  structure.   Marginally  {X  }  and  {Y  } 

are  univariate  TEAR(l)  processes,  since  the  Y  *s  do  not  appear  in  the 

£+1 
definition  of  the   X  's   and  vice  versa.    Moreover,  '■wCO  =  ai  i 

=  tyv{2)*    2  -   0,1,2,,*».   Therefore,  there  is  no  negative  correlation 

nor  any  way  to  uncouple  the  bivarlate  process  into  two  independent 

univariate  TEAR(l)  processes  unless   a--  =0.   Of  course,  if  a..   =  0, 

there  is  then  no  dependence  in  the  marginal  processes  either.   The 

bivarlate  process  with  a. .   j*  0  is   shown  in  the  next  section  to  have 

Moran's  Bivarlate  Exponential  distribution. 

Finally,  we  observe  that  if   aii+ai2+a21  =  1   ln  (57)'  we  have 

iw(0)  =  0.   Thus,  the  pairs   (X  ,Y  )   are  uncorrelated  for  all  n  and 
at  n  n 

yet  the  processes   {X  }   and   {Y  }   are  not  at  all  independent.   This 
can  be  seen  from  the  fact  that  ^(1)  =  a.~   *  °  and  ^YX^  =  a21  *  °' 
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5. ill.  Correlated  Innovations  and  Random  Coefficients. 
If  there  Is  dependence  from  both  the  Innovation  and  cross-coupled 
attenuation,  then  a  general  expression  for  (4.5)  becomes 


<(a21gll^^g12^^n, 21^12,22^11, 22^12,2l)^1^11^12>^1^1^22^ 
^  1  -  ial^2iHx2afill)   -  (*12i214alli22) 

(5.8) 


The  interesting  point  to  be  made  here  is  that  the  effects  are  additive 
in  the  numerator  of  (5.8).  This  general  model  is  called  the  BEAR(l) 
model. 

6.  NUMERICAL  EXAMPLES  OF  COVARIANCE  MATRICES. 

The  form  of  T(l)  given  in  (4.20)  can  be  misleading.  Since  the 
form  of  r(£)  is  analogous  to  that  of  a  univariate  AR(1)  process,  one 
might  think  that  the  elements  of  T(t)  should  experience  a  geometric 
decay  as  \i\  increases.  In  fact,  auto-  and  cross-covariances  can 
exhibit  many  different  patterns  depending  on  the  innovation  and  the 
attenuation  structures  of  the  model.  Some  illustrative  examples  follow. 

In  Table  6.1  an  example  is  described  where  ^yv^)  is  given  by 
(4.5).  Although  the  auto-covariance  functions  decrease  monotonlcally, 
it  is  not  at  all  like  a  geometric  decay. 

In  Table  6.2,  an  example  corresponding  to  ^wC^)  as  given  in  (5.2) 
is  exhibited.  This  case  demonstrates  that  negative  correlation  in  the 
bivariate  innovation  can  create  negative  cross-covariances  at  lag  zero. 
It  can  also  cause  oscillation  in  the  values  of  the  auto-  and  cross- 
covarlance  functions  at  subsequent  lags.   Also,  note  for  this  example 
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that  simply  by  interchanging  a^  for  aJ2  and  cu.  for  a99 .  we  can 
create  another  blvariate  series  with  the  same  r(0),  but  with  auto- 
covariance  functions  ^yvC^)  ^d  TyY^  tha.t  are  almost  geometrically 
decaying . 

In  Table  6.3,  we  give  a  final  example  corresponding  to  "fyvC^)  from 
(5.3).  It  is,  in  fact,  the  example  given  for  K  in  (5.5).  Again, 
nryY(0)  is  negative.  However,  note  that  although  the  auto-covariance 
functions  are  generally  decreasing,  the  movement  is  not  monotone  as  in 
Table  6.1,  and  certainly  not  decreasing  geometrically. 

It  is  apparent  from  all  examples  that  as  |l|  increases,  T(l) 
approaches  the  zero  matrix.  In  fact,  since  the  eigenvalues  of  the 
matrix  are  less  than  one,  the  components  of  !*(£)  all  decrease  to  zero. 


TABLE  6.1  Covariance  Matrix  Corresponding  to  (4.5) 


«„  =  .2 

a12  =  7 

a21  =  .3 

a22  =  5 

vo 

*„(«> 

t^d) 

7YY(') 

1.0000 

0.5942 

0.5942 

1.0000 

0.6159 

0.5971 

0.8188 

0.6783 

0.5412 

0.4833 

0.6386 

0.5848 

0.4466 

0.4040 

0.5371 

0.4840 

0.3721 

0.3360 

0.4462 

0.4031 

0.3096 

0.2796 

0.3714 

0.3354 

0.2577 

0.2327 

0.3091 

0.2791 

0.2144 

0.1936 

0.2572 

0.2323 

0.1784 

0.1611 

0.2140 

0.1933 

0.1485 

0.1341 

0.1781 

0.1609 

0 

1 

2 
3 
4 
5 
6 
7 
8 
9 
10  0.1236  0.1116  0.1482  0.1339 


25  0.0089  0.0081  0.0106  0.0097 


ii/t/oc  irk  i  CTBTc/ntrwAi  n 


TABLE  6.2  Covar lance  Matrix  Corresponding  to  (5.2) 


«n  =  .05 

a12  =  .60 
a   = 

a21  =  .40 
-.6449 

«22  -  01 

I 

W> 

ynW 

'»<*>      ■ 

v(l) 

0 

1.0000 

-0.1411 

-0.1411 

1.0000 

1 

-0.0347 

0.3986 

0.5929 

-0.0464 

2 

0.2374 

-0.0099 

0.0018 

0.2367 

3 

0.0059 

0.0949 

0.1421 

0.0031 

4 

0.0572 

0.0033 

0.0090 

0.0569 

5 

0.0048 

0.0229 

0.0346 

0.0042 

6 

0.0140 

0.0021 

0.0043 

0.0139 

7 

0.0020 

0.0056 

0.0086 

0.0019 

8 

0.0035 

0.0009 

0.0016 

0.0035 

9 

0.0007 

0.0014 

0.0022 

0.0007 

10 

0.0009 

0.0002 

0.0005 

0.0009 

TABLE  6.3  Covariance  Matrix 

Corresponding  to 

(5.3) 

"ll  =  ■* 

a12  =  .60 

"21  =  -40 

a22=  M 

a11.22=  "-1 

a12.22  =  '15 

a11.22  =  '1875 

a12.21=  ° 

€ 

W> 

in«) 

*„(«> 

>yy(*) 

0 

1.0000 

-0.1225 

-0.1225 

1.0000 

1 

0.1765 

0.3694 

0.5694 

0.2010 

2 

0.2659 

0.1629 

0.2629 

0.2780 

3 

0.1642 

0.1470 

0.2325 

0.1747 

4 

0.1293 

0.1024 

0.1629 

0.1367 

5 

0.0938 

0.0773 

0.1227 

0.0993 

6 

0.0698 

0.0568 

0.0903 

0.0739 

7 

0.0516 

0.0421 

0.0669 

0.0546 

8 

0.0382 

0.0312 

0.0495 

0.0404 

9 

0.0282 

0.0231 

0.0366 

0.0299 

10 

0.0209 

0.0171 

0.0271 

0.0221 
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7.  THE  MARGINAL  PROCESS. 


Writing   K  =  t(Kn)   = 


all    a12 


°21    ^ 


we  have  from  (4.20)  the 


relationship 


r(«)  ■  K*r(o) 


(7.1) 


Further,  since  K  is  a  square  matrix,  the  well-known  Cay  lay- 
Hamilton  Theorem  proivdes  that  there  exists  constants  $.  and  $„. 
such  that 


K2  -  ^K  -  ^1  =  0. 


Applying  this  result  to  (7.1)  yields 


r(«)  -  +.r{t-i)  -  *0r(«-2)  =  o.  *  *  2. 


(7.2) 


The  difference  Equation  (7.2)  applies  to  each  element  of  the  matrix 
T(£).  Hence,  for  example,  the  autocorrelation  function  of  {X  } 
satisfies 


pxx(€)  "  ♦ipnc(M>  "  Vxx(*"2)  =  °-     e  12' 


Thus,  in  general.   {X  }   has  the  correlation  structure  of  an  ARMA(2,1) 
process.   In  addition,  the  lag  1  autocorrelation  may  be  derived  from 
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(7.1)  when  i   =  1.   In  particular,  "^(l)  ■  "n^xx^  +  a12rYK^'      and 
since  ^(0)  =  ^(0)  =  1.  we  find  that  P^O)  =  an  +  ai2pYX^0^' 

Further,  the  constants  $.  and  $„  may  be  obtained  in  terms  of  the 
elements  of  K.  It  may  be  shown  directly  that  ^  =  (a^+a^)  and 
$0  =  (a19a9l-a..  jdrt,,).  Thus,  in  summary,  the  autocorrelation  functions 
for  the  marginal  processes  {X _},  {Y  }  satisfy 

p(2)   =  (an+a22)p(£-l)  +  (a12a21-ana22)p(e-2).  (7.3) 

for  i  2   2.  and 

PXX(1)  =all  +  a12PYX(0)l  (74) 

Pyy(1)  =  «22  +  a21pXY(0)'  (75) 


In  general,    {X  )    and  {Y  }   have  correlation  structures  of 

ARMA(2.1)  processes.   In  any  particular  case,  their  actual  form  will 

depend  upon  the  values  {a  }.  The  marginal  processes  will  have 
ARMA(p.q)  correlations,  where  p  i  2  and  q  <,  1.  We  illustrate  these 
ideas  with  some  examples. 

(i)  Using  a.0  =  a9.  =  0  results  in  the  X  and  Y  processes  being 

independent   TEAR(l)  processes.     From   (7.4)   and   (7.5). 

PXX^  =  all*  Pyy^1)  =  a22'  and  usinS  (7.3)  we  obtain 
Pxx(€)  =  0--  and  Pyv(^)  =  aoo'  ^he  bivariate  system  has 
become 


X  =  K^X  .  +  (l-a.JE 
n    11  n-1   v   ll7  n. 
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Y  =v(*) 


n  «  "SS'Vl  +  (1"a22)En- 


where  P(k[^=1)  =  an     and  P(K^=1)  =  a^.   and  K^n)   and 

KjL/  are  independent,  as  are  E   and  E' . 
£-*z  n       n 

(ii)  Using  a..  =  a^  =  0  leads  to  coupled  processes,  each  with 
AR(1)  correlation  structure.  Now,  PvvO)  =  a\op  and 
PyyCl)  =  a2ip>  wner©  P  =  Pw(0)-  From  (7.3)  we  may  derive 


Pxx(2k)  =  (a12a21) 
Pxx(2k+1)  =  (al2a2l)\2p 


k  =  0.1. 


This  is  the  formulation  used  by  Gaver  and  Lewis  (1980)  to 
derive  negatively  correlated  processes.  In  effect,  they  chose 
a-n  —  <i«i  =  o '.,   and  so 


Px(k)  = 


k 
a   p 


k  even 
k  odd. 


Now,  from  (5.8)  or  (5.2)  with  the  choice  of  K  used  here, 
p  =  Pyv(0)  -  (l-a)Prz:>{Q)/{l+a)'  ^nd  choosing  E  and  E*  to 
be  negatively  correlated  results  in  a  sign-switching 
autocorrelation  for  (X  }  and  {Y  } . 
(iii)  Using  a~.  =  a21  =  0  may  yield  a  process  with  ARMA(l.l) 
correlation  structure.  The  bivariate  system  is  now 
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Yn  =  4S)Yn-l  +  <l-°22>En-  <7<5> 

From  (7.4)  and  (7.5)  we  have  PwO)  =  a12p  and  PyyO)  ~  aoo' 

where  p  =  Pyv(O) •   as  before.   Further,  Equation  (7.3)  reduces 

I 
to  p(i)   =  a22p(«-l),  I  I   2.   Hence,  py^)  =  a^.  £  £  0.   and 

PXX^  =  ^a12p^ct22  '  *  *  1#   Thus«   ^Yn)   is  a  Process  wit« 
AR(1)  correlation  structure,  as  may  be  seen  directly  from 

(7.6).    In  addition.   {X  }   is  a  process  with   ARMA(l.l) 

correlation  structure,  provided   Pyy(O)  j*  0.   A  particularly 

simple  way  to  achieve  this  is  to  take   £'  =  £  .   The  process 

given  by  (7.6)  is  then  very  closely  related  to  the  EARMA(l.l) 

and  DARMA(l.l)  processes  of  Jacobs  and  Lewis  (1977,1983).   On 

the  other  hand,  we  can  choose  E'   negatively  correlated  with 

E  ,    e.g.  ,  its  antithetic,  and  may  then  induce  negative 

correlation  in  {X  }. 

(iv)  In  order  to  generate  the  autocorrelation  function  corresponding 

to  an  AR(2)  process,  it  is  necessary  that   p(£)   satisfy  the 

difference  Equation  (7.3)  for  i   =  1  also.  Using  p(-l)  =  p(l) 

this  condition  becomes  p(l)  =  ♦1/(l-$«).   Rewriting  this  with 

the  appropriate  values  for  $. ,  $9,   and  using  the  definition 

of  pYY(l)  given  by  (7.4)  yields  the  condition 


XX 


g22  +  all(g12g21~gllg22)  (7  7) 

P         g12(1+allg22-g12g21> 
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where   p  =  Pvv(O).   as  before.   In  a  model  in  which  X   and 
Y   are  independent  the  condition  becomes 


°22  +  "ll^l^l^ll0^)  "  °*  (78) 

In  this  latter  case,  the  condition  (7.8)  may  be  satisfied  by 

choosing  either:     (a)  a..   =  1;    or    (b)  a9S>  =  0    and 

a^oc  „a_  =  0.   Case  (a)  leads  to  example  (i)  above,  i.e.,  two 

Independent  processes,  each  with  AR(1)  correlation  structure, 

one  of  which  is  degenerative,   X  =  X  ,.   Case  (b)  leads  to  a 

n    n-i 

variety  of  possible  models  similar  to  those  discussed  in  the 
previous  examples,  and  one  we  have  not  noted  yet.  a  process 
with  Moving  Average  of  order  1,  MA(1).  correlation  structure. 
If  a. .    =  a~.   =  a~r>  -   0,  the  bivariate  process  is 


X  =  K^Y  1  +  (l~«,o)E 
n    12  n-1   *   12'  n 


Y  =  E\ 
n    n 


Thus,  if   E  =  E',   then   {X  }   has  the  correlation  structure 
n    n         l  n' 

of  a  MA(1)  process  and    {Y  }    is  a  sequence  of  i.i.d. 
exponentials. 
Hence,  we  cannot  derive  an  AR(2)  structure  using  (7.8).   We  require 

dependence  between   X    and   Y    and  the  satisfaction  of  condition 

^  n         n 

(7.7).  Since  the  relationship  between  Pyv^0)  and  PEE'^  given  by 
(5.2)  and  the  condition  (7.7)  are  fairly  complex,  we  shall  not  attempt 
any  general  analysis.  We  note  only  that  it  is  certainly  possible  and 
give  an  example. 
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If    (E  ,E')    have  an  Instantaneous  correlation  of  0.5,  then 
v  n  n' 

p  =  4/17.   from  (5.2).  and  (7.7)  is  satisfied  by   an  =  0.  a^   =  0.5. 

a01  =  0.3   and   aQO  =  0.1.   This  choice  of   K   and  the  correlation 

between   E    and  E'   yields  a  process   {X  }   whose  autocorrelation 
n        n  nJ 

function  satisfies 


Pxx(*)  =  0.1Pxx(«-l)  +  0.15Pxx(«-2). 


for  all  £  ±  1.  In  passing,  we  may  note  that  {Y  }  in  this  process  has 
the  correlation  structure  of  an  ARMA(2,1)  process. 

We  now  consider  the  general  case,  i.e..  the  process  with  ARMA(2.1) 
correlation  structure,  and  note  some  of  its  properties  here.   Solutions 

of  the  difference  Equation  (7.2)  depend  upon  the  behavior  of  the  roots 

2 

of  the  quadratic  equation  Z  -  4>.Z   -  $_  =  0.   It  is  easily  verified  that 

with  our  restrictions  on  ia4*}  we  have:  $  +  $  <  1,  0  <  $  <  2  and 
-1  <  $2  <  1.  These  ensure  that  the  roots  of  the  quadratic  lie  within 
the  unit  circle  (See  Box  and  Jenkins.  1976,  p.  58-59.  for  a  discussion 

of  this).   However,  we  may  also  demonstrate  that  the  roots  are  real. 

o 

The  condition  for  this  is  ^  +  4*2  I  0.     and 


♦l  +  4*2  =  (°tll+a22)2  +  4(a12a2l"alla22) 


=  (all"a22)2  +  ^lftl  *  ° 


Figure  7.1  shows  the  region  in  which  (^.^o)  will  lie  for  these 
processes.  It  is  worth  comparing  this  region  with  the  more  general  one 
available  for  a  stationary  process  with  an  AR(2)  component.   It  is  given 
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in  Box  and  Jenkins  (1976.  p.  59).  and  is  defined  by  4>.  +  $2  <  1. 
^-  -  ^2  <  -1.  -1  <  ^2  ^  ^-  *n  our  case,  only  positive  $.  is  possible, 
and  the  characteristic  quadratic  has  only  real  roots.  The  most 
important  effect  of  this  latter  property  is  that  the  autocorrelations 
are  all  positive  if  p(l)  is  positive.  We  have  seen  in  the  examples 
above  that  this  restriction  is  not  necessary  in  the  cases  when  the 
autoregressive  component  is  first  order.  Also,  we  can  introduce 
negative  autocorrelation  at  lag  one  using  (7.4)  and  (7.5)  and  negatively 

correlated   (E  ,E').   This  may  persist  for  higher  lags  depending  on  $, 
n  n  l 

and  $~  as  in  the  numerical  examples. 


Figure  7.1.   Region  of  (♦,.$«)  for  the  bivariate  exponential  process. 
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8.   JOINT  DISTRIBUTION  OF  (X  .Y  ) 

n  n 

We  turn  our  attention  to  the  formulation  of  the  joint  distribution 
of  (X  ,Y  )  by  investigating  the  joint  Laplace-Stieltjes  transform. 
For  the  Initial  bivariate  model  described  in  (3.1)  and  (3.2).  the 
expression  for  the  transform 


<PX  Y  (s.t)  =  «{exp(-sXn-tYn)}.  (8.1) 

n  n 


is  very  complicated. 

However,  two  special  cases  are  easily  verified.  If  K\r,  =  KAi  =  0 
for  all  n  in  (3.1)  and  (3.2),  we  have  after  considerable 
simplification 


*>»<■•*>-  fejte)  •  <8-2> 


This  tallies  with  the  result  of  Section  3  that  under  these  conditions 


k  2 

with  correlation  structure  Pxl(k)  =  a-.,  the  second  with  PqoC*)  ~  aoo- 


{X  }  and  (Y  }  are  independent  univariate  TEAR(l)  processes,  the  first 
i  correlation  structure  Pn(k)  =  ai  i  •  tl 
If.  in  addition,  K^   =  K^  =  Kr  and 

1    w.p.  a 


K  = 
n 


(8.3) 
0    w.p.   1-ct 


then  the  distribution  of   (X  ,Y  )   turns  out  to  be  Moran's  Bivariate 

v  n  n' 

Exponential  distribution.  Using  (3.1)  and  (3.2),  we  have  from  (6.1) 
*X  Y  (s>t)  =  '(cxPC-8<KnXn-l+(1"a)En>  "  ^Vl*0^"]  '  (8'4) 
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Using  the  Independence  of  {E  }  and  {E*},   the  stationarity  of  {X  ,Y  } 
and  conditioning  on  K  ,  we  have  after  simplification 


*fr<8-t>  "  (l+,)(llt)  -ast  •  <8'5) 


Equation  (8.5)  is  the  Laplace-Stieltjes  transform  for  the  Moran 
Bivariate  Exponential  Distribution  as  given  in  Johnson  and  Kotz  (1970, 
p.  267). 

9.  CONCLUSIONS 

The  NEAR(2)  model  has  been  demonstrated  to  yield  a  Bivariate  First 
Order  Autoregressive  process  with  exponential  marginals  by  cross- 
coupling  and  auto-coupling  the  two  marginal  processes.  The  process  has 
the  same  correlation  structure  as  the  Gaussian  Bivariate  AR(1)  process. 
The  results  for  another  bivariate  process  with  exponential  marginals 
that  was  proposed  by  Raftery  (1982)  using  the  NEAR(l)  structure  of 
Lawrance  and  Lewis  (1981)  hold  only  in  very  special  cases.  Even  then, 
as  we  have  shown,  the  correlation  structure  is  identical  to  the  Gaussian 
AR(1)  model. 

The  possibility  of  negative  correlations  was  explored  using  the 

ideas  of  correlated  innovation  and/or  cross-correlated  attenuation. 

These  situations  occur  frequently  in  modelling  physical  phenomena.   For 

example,  the  same  shock  at  time  n  to  a  system  produces  related  effects 

in  components  given  by  X   and  Y  .   Likewise,  if   X   and  Y   are 

flows  in  a  river  at  two  different  points  along  a  bank,  then   E  =  E' 

n    n 

represents  a  common  phenomenon  driving  both  series.   Finally,  when  both 
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series  are  T£AR(1)  series  linked  by  a  common  attenuation  and  Independent 

innovations,  the  bivariate  distribution  of  (X  ,Y  )  was  shown  to  be  the 

n  n 

familiar  Moran  distribution. 

Taken  together  these  options  include  numerous  possibilities  to  model 
exponential  bivariate  time  series.   There  is  still  much  work  in 
parameter  estimation  before  these  models  can  be  widely  applied.   They 
certainly  lend  themselves  at  this  time  to  an  analysis  via  simulation. 

One  other  detail  which  could  extend  the  utility  of  the  model  needs 

to  be  pointed  out.   This  is  that   Y  ,   in  (3.1)  could  be  replaced  by 

Y   .,   and   X  ,   in  (3.2)  could  be  replaced  by   X   „..   The  model  is 
n-€  n-1      v   '  r  j        n-« 

still  well  defined  and  has  obvious  physical  interpretation.   Properties 
of  this  extension  of  the  model  will  be  addressed  elsewhere. 
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